Probe-based Rapid Hybrid Hyperspectral and Tissue Surface Imaging Aided by Fully Convolutional Networks

Tissue surface shape and reflectance spectra provide rich intra-operative information useful in surgical guidance. We propose a hybrid system which displays an endoscopic image with a fast joint inspection of tissue surface shape using structured light (SL) and hyperspectral imaging (HSI). For SL a miniature fibre probe is used to project a coloured spot pattern onto the tissue surface. In HSI mode standard endoscopic illumination is used, with the fibre probe collecting reflected light and encoding the spatial information into a linear format that can be imaged onto the slit of a spectrograph. Correspondence between the arrangement of fibres at the distal and proximal ends of the bundle was found using spectral encoding. Then during pattern decoding, a fully convolutional network (FCN) was used for spot detection, followed by a matching propagation algorithm for spot identification. This method enabled fast reconstruction (12 frames per second) using a GPU. The hyperspectral image was combined with the white light image and the reconstructed surface, showing the spectral information of different areas. Validation of this system using phantom and ex vivo experiments has been demonstrated.Comment: This paper has been submitted to MICCAI2016 on 17 March, 2016, and conditionally accepted on 2 June, 201

Flexible multimode endoscope for tissue reflectance and autofluorescence hyperspectral imaging

A dual reflectance and autofluorescence spectral imaging probe compatible with the biopsy channels of standard flexible endoscopes is demonstrated. Spatially-resolved haemoglobin and autofluorescent signals from porcine bowel were obtained in vivo

Inference of tissue haemoglobin concentration from Stereo RGB

Multispectral imaging (MSI) can provide information about tissue oxygenation, perfusion and potentially function during surgery. In this paper we present a novel, near real-time technique for intrinsic measurements of total haemoglobin (THb) and blood oxygenation (SO 22 ) in tissue using only RGB images from a stereo laparoscope. The high degree of spectral overlap between channels makes inference of haemoglobin concentration challenging, non-linear and under constrained. We decompose the problem into two constrained linear sub-problems and show that with Tikhonov regularisation the estimation significantly improves, giving robust estimation of the THb. We demonstrate by using the co-registered stereo image data from two cameras it is possible to get robust SO 22 estimation as well. Our method is closed from, providing computational efficiency even with multiple cameras. The method we present requires only spectral response calibration of each camera, without modification of existing laparoscopic imaging hardware. We validate our technique on synthetic data from Monte Carlo simulation and further, in vivo, on a multispectral porcine data set

Surgical spectral imaging

Recent technological developments have resulted in the availability of miniaturised spectral imaging sensors capable of operating in the multi- (MSI) and hyperspectral imaging (HSI) regimes. Simultaneous advances in image-processing techniques and artificial intelligence (AI), especially in machine learning and deep learning, have made these data-rich modalities highly attractive as a means of extracting biological information non-destructively. Surgery in particular is poised to benefit from this, as spectrally-resolved tissue optical properties can offer enhanced contrast as well as diagnostic and guidance information during interventions. This is particularly relevant for procedures where inherent contrast is low under standard white light visualisation. This review summarises recent work in surgical spectral imaging (SSI) techniques, taken from Pubmed, Google Scholar and arXiv searches spanning the period 2013–2019. New hardware, optimised for use in both open and minimally-invasive surgery (MIS), is described, and recent commercial activity is summarised. Computational approaches to extract spectral information from conventional colour images are reviewed, as tip-mounted cameras become more commonplace in MIS. Model-based and machine learning methods of data analysis are discussed in addition to simulation, phantom and clinical validation experiments. A wide variety of surgical pilot studies are reported but it is apparent that further work is needed to quantify the clinical value of MSI/HSI. The current trend toward data-driven analysis emphasises the importance of widely-available, standardised spectral imaging datasets, which will aid understanding of variability across organs and patients, and drive clinical translation

Multisensor perfusion assessment cohort study: Preliminary evidence toward a standardized assessment of indocyanine green fluorescence in colorectal surgery

Background: Traditional methods of assessing colonic perfusion are based on the surgeon's visual inspection of tissue. Fluorescence angiography provides qualitative information, but there remains disagreement on how the observed signal should be interpreted. It is unclear whether fluorescence correlates with physiological properties of the tissue, such as tissue oxygen saturation. The aim of this study was to correlate fluorescence intensity and colonic tissue oxygen saturation. Methods: Prospective cohort study performed in a single academic tertiary referral center. Patients undergoing colorectal surgery who required an anastomosis underwent dual-modality perfusion assessment of a segment of bowel before transection and creation of the anastomosis, using near-infrared and multispectral imaging. Perfusion was assessed using maximal fluorescence intensity measurement during fluorescence angiography, and its correlation with tissue oxygen saturation was calculated. Results: In total, 18 patients were included. Maximal fluorescence intensity occurred at a mean of 101 seconds after indocyanine green injection. The correlation coefficient was 0.73 (95% confidence interval of 0.65–0.79) with P < .0001, showing a statistically significant strong positive correlation between normalized fluorescence intensity and tissue oxygen saturation. The use of time averaging improved the correlation coefficient to 0.78. Conclusion: Fluorescence intensity is a potential surrogate for tissue oxygenation. This is expected to lead to improved decision making when transecting the bowel and, consequently, a reduction in anastomotic leak rates. A larger, phase II study is needed to confirm this result and form the basis of computational algorithms to infer biological or physiological information from the fluorescence imaging data